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Abstract 

The Encircled Energy Fraction and its quantiles, no¬ 
tably the Half Energy Width, are routinely used to 
characterize the quality of X-ray optical systems. 
They are however always quoted without a statis¬ 
tical error. We show how non-parametric statistical 
methods can be used to redress this situation, and 
we discuss how the knowledge of the statistical error 
can be used to speed up the characterization efforts 
for future X-ray observatories. 

1 Introduction 

One of the parameters used to characterize the per¬ 
formance of optics is the diameter of the region of the 
focal plane containing a certain fraction of the total 
power transmitted by the optics. In X-ray optics the 
diameter of the disk containing 50% of the photons 
collected in the focal plane is commonly used, and 
it is called the Half Energy Width (HEW); at other 
wavelengths this is known as the Half Power Disk. 

The HEW is a key performance indicator of the 
angular resolution of the optics, and very often it is 
the only performance indicator used to describe the 
quality of the optics. Comparisons between different 
X-ray mirror technologies are also often done solely 
in terms of achieved HEW. 

To our knowledge, nobody ever quotes the statis¬ 
tical error associated with the HEW, nor any confi¬ 
dence band is given when the Encircled Energy Frac¬ 
tion (EEF) is plotted. Still, this is a natural thing 


to do when measurement results are reported. In¬ 
deed it is almost always the case that the functional 
form of the point spread function of an optics is not 
known, and therefore there is no ready-to-use for¬ 
mula that can be applied to bound the experimental 
values measured. This observation however suggests 
that a solution to the problem should be looked for 
in non-parametric statistical methods. 


In this article we show that by making use of text¬ 
book non-parametric statistics solutions, the statis¬ 
tical error and the confidence interval for a measure 
HEW value, and the confidence band of the EEF can 
be easily determined. 


The remainder of this article is organized as fol¬ 
lows. In § [^ we describe the experimental config¬ 
uration considered, and recast the notion of EEF in 
terms of the empirical Cumulative Distribution Func¬ 
tion (CDF); once this is done the conhdence band of 
the EEF can be easily calculated. In §[^we continue 
mapping terms known in optics to terms used in prob¬ 
ability and statistical theory; again this will lead in 
a straightforward manner to calculate the standard 
error of the HEW. The logical extension to the cal¬ 
culation of a confidence interval for the HEW is illus¬ 
trated in § [^ The solutions discussed are illustrated 
by means of numerical experiments in §[^ and further 
discussed in § [^ Conclusions are drawn in § [^ 
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and 


2 Confidence band for the en¬ 
circled energy fraction 


Consider an experimental set up designed to char¬ 
acterize the optical performance of an imaging sys¬ 
tem. We limit our discussion to a photon counting 
experiment. Without loss of generality we can limit 
ourselves to the case where the source of photons is 
on-axis and monochromatic. Photons are reflected 
by the optical system and collected at the focal plane 
with an ideal position-sensitive detector. 

For a certain measurement let n be the number of 
photons detected. Let {Ri}i=i...n be the distances 
of the detected photons from the center of the fo¬ 
cal plane, the place where a perfect imaging system 
would image a point source at infinity. The empirical 
Cumulative Distribution Function (CDF) is defined 
as 

1 ” 

Fn{r) = -Y.m<r), ( 1 ) 

1 

where 


I[R, < r) 


1 ii Ri <r 

0 otherwise. 


( 2 ) 


It is obvious that the CDF is equivalent to the EEF, 
and that F is therefore an estimate of the true EEF. 
That is to say, F is an estimate of the probability that 
a certain measured radial distance R be less than r: 


F{r) = P{R < r ), (3) 

Once we recognize the nature of the EEF, it is imme¬ 
diately possible to calculate a confidence band for it 
by invoking the Dvoretzky-Kieer-Wolfowitz inequal¬ 
ity m, that states that for any e > 0 

P (^sup |E(r) - F{r)\ > e) < . (4) 

If we now take [2] 

4 = log(2/a)/(2n) (5) 


U{r) = mm{Fn{r) + en,l} , (7) 

by substituting in Equation @ we obtain 

P {L{r) < F{r) < U{r)) > 1 — a ,yr . (8) 

This equation defines for all r the 1 — a confidence 
band for the EEF. Examples of EEF confidence bands 
calculated in this manner are shown in Figure (see 
§[^for the details). 

3 The standard error of the half 
energy width 

By definition the HEW is twice the 2-quantile (the 
median) of the CDF, formally 

H = 2F-\l/2). (9) 

While the confidence band calculated in the pre¬ 
vious section can be used to gain an idea of the un¬ 
certainty associated with the estimate of the HEW, 
Equation (|^ cannot be strictly inverted: that is, 
knowledge of the confidence band of the EEF cannot 
be used to infer the confidence interval for the HEW. 
Instead we need to know the variance and the distri¬ 
bution of the HEW. The bootstrap method Hi] can 
be used to arrive at the result. 

The bootstrap is a statistical method that derives 
information about the variance and the distribution 
of any statistics using only the data available. Text¬ 
books discuss the method in contexts removed from 
optics: in the following we follow [3] (§ 5), but recast 
the description in terms commonly use in optics. 

Going back to our observed values, {Ri\i=i...n) we 
proceed as follows. 

1. Draw with replacement from the data a new se¬ 
ries of radial distances 

2. Calculate a new HEW value FI' from the new 
series. 


where a € (0,1), and define 

L(r) = max{F„(r) - e„,0} 


( 6 ) 
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3. Repeat the previous two steps B times to ob¬ 
tain the bootstrap series of HEW estimates 


4. The bootstrap variance estimate of the HEW is 
obtained by calculating the variance of the boot¬ 
strap series. 

The standard error we are after is the sample stan¬ 
dard error of the bootstrap series: 


assumption that the bootstrap distribution is an un¬ 
biased realization of the true HEW distribution. 

With a bit more work one can generate a better 
confidence interval, better meaning that its coverage 
of the real confidence interval is more accurate. In 
the following section we describe how this is done. 


se 



B- 1 


( 10 ) 


where H' = J2ki^k/^) ■ The issue is now how large 
B should be in order to provide a sufficiently accurate 
estimate of the variance. In the literature values of B 
of the order of 200 are used. However a larger value 
is required to arrive at a good estimate of bootstrap 
confidence intervals, as we discuss in the following 
section. 


4 Confidence interval for the 
half energy width 


4.2 The BCq confidence interval 


With the BCq (bias-corrected and accelerated) 
method the extremes of the confidence interval are 
again based on two quantiles of the bootstrap CDF, 
but these are calculated in a manner that ensures that 
the resulting confidence interval has a higher proba¬ 
bility of overlapping with the true one. 

The two quantiles used are: 
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Once the standard error has been determined, we can 
make use of other non-parametric techniques to es¬ 
timate the confidence interval of the HEW. In the 
literature a number of approaches are available to 
this end. Here we discuss two of them: the percentile 
method, and the so-called BCa method. Both meth¬ 
ods make use of percentiles of the cumulative distri¬ 
bution of the HEW bootstrap series, but they differ in 
the manner in which those percentiles are calculated. 
We follow [5]. 

4.1 The percentile confidence interval 

The percentile method makes use of the bootstrap se¬ 
ries of HEW estimates to calculate the boundaries of 
the confidence interval. Let G be the empirical CDF 
of the bootstrap series {H'^}k=i...B obtained above. 
The 1 — a percentile confidence interval for the HEW 
is delimited by the two percentiles of G, G“^(a/2) 
and G“^(l — a/2). It is important to note that the 
confidence interval obtained will not necessarily be 
centered on the HEW estimate. 

This method is straightforward to implement, re¬ 
quires B to be of the order of 2000, and makes the 


where $(.) is the Normal CDF, and a and zq are 
two constants that must be computed from the data. 
How is explained in detail in § [A| 


5 Numerical experiments 


We illustrate the issues discussed in this article with 
the results of numerical simulations. 

We consider the case of optics that have a point 
spread function that can be represented as a circular 
bivariate Normal: 

P{x,y) = . (13) 

The HEW corresponding to this function can be cal¬ 
culated analytically and is 

HEW = 2 V 2 In 2 = 2r5o , (14) 


where r^Q « 1.177 is the median of the radial distri¬ 
bution obtained from Equation (131. 

In Figure we show the 90% confidence bands 
calculated with Equation ([^ for two realizations 
of Equation containing 100 and 1000 photons 
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respectively. As can be expected the confidence 
band becomes narrower as the number of photons 
increases. Note how relatively few photons are suffi¬ 
cient to tightly bound the EEF. 


100 photons 



radial distance [AU] 


1000 photons 



radial distance [AU] 


Figure 1: Two realizations of the 90% confidence 
band of the encircled energy fraction for an imag¬ 
ing system with a point spread function equal to a 
bivariate circular Normal. As can be expected, the 
confidence band narrows around the measured data 
with increasing number of photons. 


In a second numerical experiment, we have ran¬ 
domly drawn 500 photons according Equation (13). 
For each of the 5000 realization we have calculated 
a new HEW, and the distribution of HEW values is 
shown in Figure this is meant to represent the 
underlying true distribution of the infinite popula¬ 
tion of HEW measurements. Another 500 photons 
were then drawn at random to represent the mea¬ 
sured data, and this latter dataset is used to cal¬ 
culated the measured HEW and the corresponding 
BCa 90% confidence interval, also shown in the fig¬ 
ure. This example illustrates how the distribution is 
indeed centered on the true value of the HEW (Equa¬ 
tion (14)), but that the bootstrap confidence interval, 
while obviously not perfectly centered on the under¬ 
lying distribution, does indeed give a good indication 
of the statistical uncertainty associated with the re¬ 
sult. 



Figure 2: Simulation of a 500-photon measurement 
of a bivariate circular Gaussian point spread func¬ 
tion. The curve shows the true HEW distribution, 
obtained by simulating 5000 measurements. Also 
shown the result of one measurement, with the 90% 
confidence interval calculated with the BCa method 
and B = 2000. The exact HEW is about 2.35 (Equa¬ 
tion (1T4|. 
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As a final check on the validity of the approach, we 
remark that for a bivariate circular Normal the radial 
values are distributed according to the Rayleigh dis¬ 
tribution, for which each quantile and its confidence 
intervals can be calculated [5]. The HEW calculated 
in this manner is also shown Figure together with 
its 90% confidence interval. The two intervals are in 
good agreement, even if the confidence interval cal¬ 
culated analytically is smaller; this can be easily un¬ 
derstood because analytical methods make often use 
of the knowledge of the distribution to arrive at an 
unbiased estimator of minimum mean squared error. 
However, when the distribution is unknown this ex¬ 
ample show that the bootstrap method arrives at a 
reasonable estimate of the true confidence interval. 


6 Discussion 

We have introduced methods to calculate the confi¬ 
dence band of the EEF and the confidence interval of 
the HEW. The bootstrap method for the computa¬ 
tion of the confidence interval can be applied to any 
measurement, but for the results to be representa¬ 
tive of the true HEW confidence level, the data must 
be taken in such a manner that they randomly sam¬ 
ple the point spread function of the optics. This is 
certainly the case for full flood illumination measure¬ 
ments: then any part of the aperture has the same 
probability of being probed by X-ray photons, and 
the methods discussed here can be applied straight¬ 
forwardly. One could implement the method at a 
facility so that the statistical accuracy reached by a 
measurement can be assessed on-line, and the mea¬ 
surement stopped when a predefined accuracy is ar¬ 
rived at. It is our opinion that in many cases this 
would lead to a reduction of the required measure¬ 
ment time without any loss of measurement accuracy. 

We have assumed an ideal detector, or at least one 
with infinite spatial resolution. Real detectors will in 
general be pixellated, and may have to be read out 
at regularly intervals, so that the signal in each pixel 
is proportional to the number of photons detected. 
In this case the method can be applied by replacing 


Equation Q with 

1 "■ 

Fn{r) = ^ — '^P^I{Ri<r), (15) 

where pi is the number of photons detected by the 
pixel, and Ri is now the radial distance of the pixel 
from the optical axis. 

Not all measurement methods sample the aper¬ 
ture of the optic uniformly, and therefore the applica¬ 
tion of the bootstrap method requires that changes 
be made to the way data are collected. Consider 
for instance the characterization if Silicon Pore Op¬ 
tics (SPO) at the X-ray Pencil Beam Facility of the 
BESSY-H synchrotron [7]. The SPO have an aper¬ 
ture that is close to rectangular, and they are sam¬ 
pled with an X-ray pencil beam following a rectangu¬ 
lar grid. This sampling strategy does not cover the 
aperture uniformly, and therefore it would have to 
be changed so that the pencil beam probes the optics 
aperture at random positions. Also in this case the 
ability to asses the statistical error on the measured 
HEW is likely to deliver a significant advantage in 
terms of measurement time. 

In the case of optics built by assembling hundreds 
of individual X-ray optics, like the Athena mission [S], 
the methods presented here can be used to screen 
each individual module to a certain accuracy before 
deciding whether they should be characterized fur¬ 
ther and and how they should be eventually inte¬ 
grated in the larger telescope assembly. 

More in general, making measurement results 
available with a statistically meaningful estimate of 
their significance can only be seen as useful pursuit, 
and it is hoped that the techniques described in this 
article may encourage the use of statistical errors 
when reporting HEW results. 

7 Conclusions 

The characterization of the optical properties of X- 
ray optics makes use almost exclusively of the EEF 
and one of its quantiles (HEW). These are always 
reported without any mention of a confidence band 
or confidence interval. We have shown that straight¬ 
forward non-parametric statistical methods provide 
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ways to place a confidence band around the EEF, 
and a confidence interval around the HEW. 


A Calculation of the BCq pa¬ 
rameters 


In this section we show how the parameters required 
in the calculation of the BCa are arrived at. For 
the details refer to [3]. We consider here the original 
sample R = and a statistics of interest 9 = 

s{R) (in our case this would be the median or the 


HEW). 

The parameter zq in Equation [TT] is called the bias 
correction, and is obtained from the proportion of 
bootstrap estimates smaller than the original esti¬ 
mate: 


io = $ ^ 




(16) 


where <&(.) is the Normal CDF. 

The acceleration parameter d can be calculated in 
terms of the jackknife values of the statistics 6. Let 
be the original sample with the i-th point re¬ 
moved, 0(i) = s(.R(j)), and 0(.) = 
acceleration parameter is 




(17) 
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